The field of photography continues to develop, offering more options to a photographer. With analog photography, developed film revealed the final result of the exposure, color balance, focus, and other image capture factors as they had occurred at the time of image capture. Until recently, digital photography has been about digitizing what used to be an analog process. Digital photography affords more options to photographers than analog photography after an image is captured, as processing applications can be used to enhance and clean up the image respecting certain qualities like exposure, color balance, saturation, and others. However, if the focus of the captured digital image is off, or if the photographer wishes to shift to a different viewpoint, existing processing applications cannot correct for these factors.
Plenoptic photography comes closer to rendering the full variety of view angles and multiple focal points offered by the direct observation of objects by capturing the light field, also referred to as radiance, of a target image scene. To capture a plenoptic image, a microlens array can be mounted in front of an image sensor, effectively capturing many images of a target scene, with each image capturing a portion of the target scene from a slightly different viewpoint. As such, multiple light rays are captured from varying viewpoints for each pixel of a synthesized final image of the scene. This provides raw sensor data containing four-dimensional radiance data about each pixel point in a potential final image of the target scene: two dimensions of the spatial position of a light ray and two dimensions of the orientation of a light ray at the pixel. With software, this data can be synthesized into a final 2D or 3D image from any of the vantage points or focus points represented by the radiance data, extending the capabilities of digital photography and affording greater flexibility for photographers to alter vantage point or focus after image capture.
Plenoptic camera technology offers several imaging capabilities that may be desirable in mobile devices, including but not limited to full 3D imaging, refocusabilty, and High Dynamic Range (HDR) imaging. However, processing the plenoptic sensor data is computationally expensive, typically requiring parallel processing on central processing units (CPUs) or intensive processing on a graphics processing unit (GPU). Accordingly, plenoptic photography requires prohibitively large amounts of data storage and processing for implementation in mobile photography, given the relatively limited GPU memory of conventional mobile devices.